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Abstract —  Consider  a  communications  network  consisting  of 
mobiles,  some  of  which  can  serve  as  a  receiver  and/or  transmitter 
in  a  multihop  path.  There  are  random  external  data  processes, 
each  destined  for  some  destinations.  At  each  mobile  the  data 
is  queued  according  to  the  source-destination  pair  until  trans¬ 
mitted.  The  capacities  of  the  connecting  channels  are  randomly 
varying.  Time  is  divided  into  small  scheduling  intervals.  At 
the  beginning  of  the  intervals,  the  channels  are  estimated  via 
pilot  signals  and  this  information  is  used  for  the  scheduling  deci¬ 
sions  during  the  interval,  concerning  the  allocation  of  transmis¬ 
sion  power  and/or  time,  bandwidth,  and  perhaps  antennas,  to 
the  various  queues  in  a  queue  and  channel-state  dependent  way, 
to  assure  stability.  Lost  packets  might  or  might  not  have  to  be 
retransmitted.  General  networks  are  covered,  conditions  used  in 
previous  works  are  weakened,  and  the  distributions  of  the  in¬ 
put  file  lengths  can  be  heavy  tailed.  The  resulting  controls  are 
readily  implementable.  The  choice  of  Liapunov  function  allows  a 
range  of  tradeoffs  between  current  rates  and  queue  lengths,  under 
very  weak  conditions.  Because  of  the  non-Markovian  nature  of 
the  problem,  we  use  the  perturbed  Stochastic  Liapunov  function 
method,  which  is  designed  for  such  problems.  Extensions  con¬ 
cerning  acknowledgments,  multicasting,  non-unique  routes,  and 
others,  are  available. 

Index  terms:  Scheduling  in  stochastic  networks,  randomly- 
varying  link  capacities,  mobile  networks,  stochastic  stability,  sta¬ 
bility  of  networks  with  randomly  varying  links,  routing  in  ad-hoc 
networks,  perturbed  stochastic  Liapunov  functions,  heavy  tailed 
distributions. 

I.  Introduction 

Consider  a  network  of  M  mobiles  or  nodes.  There 
are  S  external  sources  of  bursty  data  processes,  each 
source  having  a  unique  entry  and  destination  node  in 
the  system.  In  this  short  paper,  the  routing  is  a  priori 
fixed  for  each  of  the  source-destination  pairs.  This  sim¬ 
plifies  the  notation  considerably,  but  is  not  necessary 
for  the  method  to  work  [8] .  At  each  mobile  the  data  is 
queued,  until  transmitted,  in  an  infinite  buffer  depend¬ 
ing  on  the  source.  We  are  concerned  with  the  allocation 
of  power  and/or  time  and  bandwidth  to  the  queues  in  a 
queue  and  channel-state  dependent  way  to  assure  sta¬ 
bility.  The  capacities  of  the  connecting  channels  are  a 
correlated  random  process.  Time  is  divided  into  small 
scheduling  intervals.  At  the  beginning  of  the  intervals. 
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the  capacities  (or  surrogates  such  as  the  S/N  ratios) 
are  estimated  via  pilot  signals  and  this  information  is 
then  used  to  make  the  scheduling  decisions  during  that 
interval.  Using  such  information  can  improve  the  per¬ 
formance  dramatically  [1] .  Owing  to  the  random  nature 
of  the  arrival  and  channel  processes,  the  computation 
or  even  the  existence  of  stabilizing  policies  is  not  at  all 
obvious.  The  approach  is  a  network  extension  of  the  de¬ 
velopment  for  the  one-node  case  in  [4].  The  references 
[5],  [8]  are  for  networks,  and  contain  many  extensions 
to  non-unique  routing,  acknowledgments  of  receipt  of 
packets  required,  multicasting,  randomly  varying  num¬ 
bers  of  users,  randomly  available  frequencies  or  oppor¬ 
tunistic  frequency  allocation,  etc.,  and  develop  methods 
for  getting  the  a  priori  routes.  This  paper  uses  a  sim¬ 
pler  Liapunov  function  perturbation,  based  on  a  simple 
mixing  condition,  that  has  many  advantages  and  allows 
us  to  deal  with  processes  not  covered  by  previous  work. 
It  is  more  manageable,  and  it  extends  the  methods  so 
that  heavy  tailed  input  processes  can  be  handled.  All 
the  extensions  in  the  references  can  be  carried  over  to 
the  heavy  tailed  case. 

Owing  to  the  non-Markovian  nature  of  the  sys¬ 
tem  state^  classical  stability  methods  cannot  be  used 
without  revision,  and  a  perturbed  Liapunov  function 
method  [4],  [7]  is  adapted  to  obtain  the  desired  results. 
With  this  method,  and  X  denoting  the  vector  of  queue 
values  (measured  in  packets)  at  all  the  nodes,  one  starts 
with  a  basic  Liapunov  function  V{X)  that  works  for 
a  “mean  flow”  system.  Then  one  gets  a  perturbation 
5V{n)  to  V{X)  so  that  V{X{n))  +  5V{n)  can  be  used  as 
a  Liapunov  function  for  the  actual  non-Markov  phys¬ 
ical  system  and  imply  the  desired  stability.  The  ac¬ 
tual  decision  rule  is  based  on  the  gradient  of  U(A)  and 
is  readily  implemented.  The  basic  result  is  that,  if  a 
certain  “mean  flow”  or  fluid  approximation  process  is 
stable,  then  so  is  the  physical  system  under  our  schedul¬ 
ing  rule.  This  stabilizability  of  the  mean  flow  approx¬ 
imation  can  often  be  readily  verified.  The  condition  is 
“nearly”  necessary  as  well. 

The  (n-l-l)st  scheduling  interval  is  called  the  nth  slot. 

^For  example,  Rayleigh  fading  is  not  Markovian]. 
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The  time  argument  n  denotes  the  beginning  of  the  nth 
slot,  and  is  referred  to  as  “time  n.”  Let  Xi^k{n)  denote 
the  queue  size  at  time  n  at  node  k  of  data  coming  from 
source  i.  If  node  k  is  not  on  the  path  for  source  i,  then 
Xi^k{n)  =  0.  Define  the  vectors  Xk{n)  =  {Xi^k{n),i  < 
S'}  and  X{n)  =  {Xk{n),k  <  M},  with  canonical  values 
Xk  and  X,  resp.  For  weights  Wi^k  >  0,  and  until  further 
notice,  the  basic  Liapunov  function  will  be^ 

V{X)  =  Y,w.,kXl„  p>2.  (1.1) 

i,k 

(1.1)  will  be  modified  slightly  for  the  heavy  tailed  case. 

Stability  is  supposed  to  assure  a  robustness  of  be¬ 
havior  to  small  changes  in  the  system.  For  this  reason, 
as  well  as  because  {X(n)}  is  rarely  Markovian,  it  is 
preferable  to  use  methods  that  do  not  require  Marko- 
vianness.  The  perturbed  Liapunov  function  method  is 
a  powerful  tool  for  such  a  purpose.  Generally,  there  are 
many  criteria  that  are  of  interest  to  each  of  the  users, 
e.g.,  mean  delay  and  variance  of  delay.  One  should  ex¬ 
periment  with  the  form  of  the  Liapunov  function  to  see 
what  the  tradeoffs  are  between  competing  criteria,  a 
procedure  that  yields  better  results  than  simply  work¬ 
ing  with  a  single  fixed  rule,  whatever  it  is. 

There  is  much  work  on  scheduling  under  various 
types  of  randomness.  But,  other  than  [4],  [5],  [8]  little  is 
available  for  the  general  network  case  when  the  chan¬ 
nels  are  randomly  varying.  For  the  one-node  case,  if 
the  rate  of  transmission  is  proportional  to  power,  then 
[1],  [10]  gets  rules  for  power  allocation  whose  form  is 
similar  to  ours  when  p  =  2  (called  “max  weight”  rules 
there),  and  which  are  based  on  stability  considerations. 
Large  deviations  estimates  and  a  Markovian  setup  are 
used.  Reference  [12]  considered  the  problem  of  dy¬ 
namic  power  allocation.  Since  the  channel-rate  and 
data-arrival  processes  are  i.i.d.  sequences,  the  range 
of  applications  is  small. 

The  papers  [2],  [3]  deal  with  related  problems,  again 
essentially  for  one-node  systems.  There  is  a  set  of 
parallel  processors,  and  the  connectivities  between  the 
sources  and  the  processors  vary  randomly.  They  prove 
results  concerning  the  limit  of  (queue  length  at  t)/t, 
and  show  that  this  limit  is  zero.  This  is  used  to  show 
that  the  integral  of  the  “rates”  of  transmission  per  unit 
time  converges.  Such  a  result  does  not  quite  imply  sta¬ 
bility  (in  our  sense)  of  the  queue  length  process,  since 
it  can  grow  sublinearly.  They  allocate  a  single  resource 
(e.g.,  bandwidth)  and  the  rate  is  proportional  to  the 
allocation.  The  work  [11],  for  a  one  node  model,  has 
a  Markovian  channel-state  process,  the  data  input  se¬ 
quence  is  i.i.d.,  and  a  “complete  resource  pooling”  con¬ 
dition  is  required.  The  decision  rule  is  the  same  as 
ours  for  a  quadratic  Liapunov  function.  The  emphasis 

^One  could  let  the  powers  depend  on  i,  k,  or  use  sums  of  ap¬ 
propriate  convex  functions  [8].  But  the  simpler  form  (1.1)  is 
adequate. 


is  on  stability  in  the  heavy  traffic  limit,  and  showing 
how  the  problem  simplifies  there.  Section  2  states  the 
assumptions.  Section  3  gives  the  stability  proof  for  the 
non-heavy-tailed  case,  and  the  changes  for  the  heavy¬ 
tailed  case  are  in  Section  4. 

II.  Assumptions 

Queue  (z,  k)  is  the  queue  for  source  i  at  node  k.  If 
the  path  for  source  i  does  not  use  node  k,  then  the 
queue  does  not  exist.  Let  k  denote  a  canonical  node, 
and  f(i,  k)  the  node  that  the  output  of  queue  (z,  k)  goes 
to.  I.e.,  queue  (z,  k)  feeds  to  queue  (z,  /(z,  k)).  If  node  k 
is  the  final  destination  for  source  z,  then  terms  involv¬ 
ing  /(z,  k)  are  ignored.  Let  b{i,  k)  denote  the  node  that 
queue  (z,  fc)  is  fed  from.  I.e.,  queue  {i,b{i,k))  feeds  to 
queue  (z,fc).  If  node  k  is  the  origin  node  for  source  z, 
then  terms  involving  5(z,  k)  are  ignored.  Let  Xn  denote 
the  minimal  a-algebra  that  measures  the  systems  data 
until  time  n  as  well  as  the  channel  state  in  slot  n.  This 
channel  state  is  available  at  time  n.  Let  denote  the 
expectation  conditioned  on  We  say  that  the  pack¬ 
ets  sent  in  slot  n  are  sent  at  time  n.  Let  di^kiji)  denote 
the  number  of  packets  sent  from  queue  (z,  k)  at  time 
n.  It  will  depend  on  the  current  channel  state  and  will 
be  a  function  of  the  resources  (e.g.,  power,  bandwidth) 
allocated  to  that  queue.  Let  ai^kin)  denote  the  random 
number  of  arrivals  in  slot  n  from  the  exterior,  if  any, 
from  source  z  at  node  k.  These  will  be  non-zero  only 
for  the  unique  node  fc(z)  at  which  source  z  enters  the 
network. 

Stability.  An  appropriate  definition  of  stability  is  a 
“uniform  mean  recurrence  time”  property  [4] ,  [5] .  Sup¬ 
pose  that  there  are  0  <  qo  <  oo  and  a  real-valued 
F(-)  >  0  such  that:  For  any  n,  and  ui  =  minjfc  > 
n  :  \X{k)\  <  <7o},  we  have^ 

En  [cti  -  rz]  <  F(A(n))/{|x(n)|>5o}-  (2-1) 

Then  the  system  is  said  to  be  stable.  The  definition  im¬ 
plies  recurrence  to  some  compact  set.  If  |Ar(n)|  reaches 
a  level  qi  >  qo,  then  the  conditional  expectation  of 
the  time  required  to  return  to  a  value  qo  or  smaller  is 
bounded  by  a  function  of  <71 ,  uniformly  in  the  past  his¬ 
tory  and  in  n.  The  right  side  of  (2.1)  depends  only  on 
X(n),  and  not  on  any  other  data,  even  though  the  chan¬ 
nel  and  arrival  processes  are  random  and  correlated. 

The  decision  rule.  The  number  of  packets  trans¬ 
mitted  from  queue  {i,k)  in  slot  n  is  di^k{n),  and  this 
depends  on  the  committed  resources.  The  assignments 
are  subject  to  constraints.  If  the  constraints  are  only 
local,  such  as  bounds  on  the  total  nodal  power,  the 
di,fc(n)  for  all  z  can  be  determined  at  node  k.  If  the 
constraints  involve  more  than  one  node  (e.g.,  if  neigh¬ 
boring  nodes  cannot  use  the  same  carrier  frequency), 

=  00,  unless  otherwise  defined. 


then  the  assignments  require  coordination  among  the 
nodes. 

As  in  classical  stability-control  theory,  the  idea  is 
to  choose  the  di^k{n)  to  minimize  EnV{X{n  +  1))  — 
V{X{n))  as  well  as  possible.  To  motivate  the  actual 
rule,  first  evaluate  EnV{X{n+l))  —  V{X{n)).  We  have 

w^,k  EnXl^in  -hi)  -  Xl^in) 

=  w^^kXf^^{n)  [-di,fc(n)  -h  E„a^^k{ri)  +  di^b{i,k){'ri)] 

+0{Xlf{n)). 

Summing  over  i  and  k  yields,  modulo  0{X^^^(n))  and 
the  “arrival”  terms, 

-  [^^^kX^i7k^{n)  -  d,^kin) 

i,k 

=  -  '^Wi^kXfj:^{n)  [di^k{n)  -  di^b{i,k){n)]  ■ 

i,k 

(2.2) 

The  0{Xff.  (n))  are  nonlinear  functions  of  di^kin)  and 
higher  conditional  moments  of  the  and  would 

be  hard  to  deal  with.  Fortunately,  as  in  [4],  it  is  enough 
to  work  with  the  term  that  is  first  order  in  the  decisions, 
those  in  (2.2). 

If  the  decisions  can  be  made  independently  at  each 
node,  then  our  decision  rule  (for  each  k)  is  a  maximizer 
in 

d^,k{.n). 

(2.3) 

If  there  are  constraints  that  involve  a  set  of  nodes,  then 
the  decisions  for  them  must  be  made  together,  and  the 
decision  rule  is  a  maximizer  in 

di^k{n), 

i,k 

. 

or,  equivalently  (by  rearranging  terms  in  (2.4)),  in 
max  y7  Wi,fcAf~^(n)  [di,fc(n)  -  (ii,6(i,fc)(n)]  . 

{di,fc(n);i,fc} 

2,fc 

(2.5) 

X{n)  is  rarely  Markovian,  so  classical  stochastic  sta¬ 
bility  theory  [6]  cannot  be  used  directly.  However,  the 
perturbed  Liapunov  function  method  [4],  [7],  [9]  will  al¬ 
low  us  to  show  that  the  rules  (2.3),  (2.4),  or  (2.5)  yield 
a  stable  system. 

Let  Lk{n)  denote  the  (vector)  set  of  channel 
states,  at  time  n,  of  all  of  the  channels  originating 
at  node  fc,  which  are  the  links  {{i,k),{i,  f{i,k))  : 
all  i  using  node  k}.  Lk{n)  could  be  simply  the  set  of 
S/N  ratios  at  the  receiver  corresponding  to  unit  trans¬ 
mitted  power.  It  is  convenient  to  work  with  the  vector 
Lfe(n),  rather  than  with  the  individual  links,  since  the 
decisions  at  each  node  k  depend  on  the  states  of  all 


of  the  outgoing  links.  Lk{n)  might  denote  other  quan¬ 
tities  besides  the  channel  quality.  For  example,  there 
might  be  power  constraints  that  vary  randomly  due  to 
interference  from  exogenous  sources.  If  some  link  at 
node  k  is  unavailable  at  time  n,  then  that  fact  could 
also  be  included  in  Lk{n).  We  suppose  that  the  range 
of  values  of  the  channel  state  vector  is  a  finite  set  for 
each  node  k.  We  use  the  (vector-valued)  symbol  j  for 
the  canonical  value  of  Lk{n),  for  any  k,  n.  The  range  of 
the  variable  j  will  depend  on  the  node  k  and  will  not  be 
specified.  Let  Ui^k{3,X)  denote  the  control  function  at 
queue  {ilk).  It  represents  the  resources  (power,  time, 
bandwidth,  etc.)  allocated  to  queue  {i,k).  Also,  unless 
otherwise  noted,  its  dependence  on  the  queues  is  only 
on  Xk  and  the  required  queue  values  at  the  immedi¬ 
ate  upstream  nodes,  namely  the  ATj  jp  for  all  i.  If 
source  i  does  not  use  node  fc,  then  ignore  Ui^k{j,X). 
The  control  Ui^k{j,X)  determines  the  amount  of  data 
that  is  sent.  Let  gi,k{j,  Xik^Ui^kij,  X))  denote  the  ac¬ 
tual  amount  of  data  that  is  sent  from  queue  {i,  k)  un¬ 
der  channel  state  j  and  control  Ui^k{').  This  defines 
di^kin);  i.e.,  the  channel  rate  for  queue  {i,k)  associ¬ 
ated  with  current  channel  state  j  =  Lk{n)  and  control 
Ui,k{j,X{n))  is  d,^k{n)  =  9i,k{j,  X,^k{n),Ui^k{j,  X{n))). 
The  Xi^k  is  an  argument  of  gi,k{')  only  because  the 
amount  sent  cannot  be  larger  than  the  queue  content. 

Assumptions.  (A2.4)  requires  that  there  are  controls 
under  which  the  mean  service  rate/slot  at  queue  (t,  k) 
for  any  i  that  uses  node  k  is  slightly  greater  than  A“. 
Similar  conditions  are  commonly  used  in  the  study  of 
the  stability  of  stochastic  networks. 

A2.1.  The  maximizing  constrained  di^k{n)  exist  and 
are  Borel  functions  of  the  {X{n),Lk{n),i,k}. 

A2.2.  There  is  a  Ki  <  oo  such  that  for  all  i, 
En\ai,k{n)7  ^  Xi.  There  are  A“j,  and  a  p{k)  which 
goes  to  zero  as  k  ^  oo  such  that  \Enai^k{l)  —  Kk\  — 
p{l  —  n),  for  all  n,l  >  n,  to,  i,  k. 

The  A“j,,  called  the  mean  external  data  arrival  rate  for 
source  i  at  node  k,  is  zero  if  k  yf  k{i).  Define  A“  = 

\,k{^)' 

A2.3.  There  are  H^,  j  >  0  such  that  H^  ^  =  1  and 
\EnI{Lk{i)=j}  -  life  J I  <  p{l  -  n),  for  all  n,  l>n,uj. 

A2.4.  Define  ATq  =  niaxi^k,j,u,x  gi,k{3^  Xi^k,Ui^k{j,  X)). 
There  is  a  control  {■Ui_fc(-);  i, /c}  under  which  the  fol¬ 
lowing  holds.  There  are  such  that  ql  k  ~ 

gi,k{EXt,k{n),Ui,k{j,X{n)))  if  Xi^k{n)  >  Kq.  Also, 
gi,k{j,  Xi^k{n),Ui,k{j,  X{n)))  <  ql  ,,  if  Xi^k{n)  <  Kq. 

“^The  lower  bound  Kq  is  introduced  only  because  if  the  queue 
content  is  smaller  than  the  maximum  of  what  can  be  transmitted 
on  a  scheduling  interval,  then  the  mean  output  might  be  too  small 
to  assure  the  —cq  value.  E.g.,  if  a  queue  is  empty,  then  there  are 
no  departures. 


The  ql  satisfy,  for  nodes  k  used  by  source  i, 

Qi,k  =  >  Aj .  (2-6) 

3 

Comments  on  the  assumptions.  (A2.2)  and  (A2.3) 
are  simply  mixing  conditions  on  the  data  arrival  and 
channel  processes,  resp.,  and  do  not  appear  to  be  re¬ 
strictive.  Let  Ilfcj  denote  the  steady  state  probability 
of  channel  state  j  at  node  k.  Then  (A2.3)  says  that 
the  conditional  probability  of  state  j  at  time  I  given 
the  data  to  time  n  converges  to  the  steady  state  value 
as  I  —  n  — >  oo.  It  holds  for  the  received  signal  power 
associated  with  Rayleigh  fading. 

By  the  definition  (2.6),  for  k  ^  k{i),  qi^b(i,k)  = 
Y.]iib(i,k)^b(i,k),3-  It  is  implied  by  (A2.4)  that  there 
is  Co  >  0  such  that  the  q^  can  be  chosen  to  satisfy 

Aj  Qi,k(i)  —  ^0;  (2.7a) 

and,  for  k  ^  k{i), 

average  into  (i,  k)  —  average  out  of  {i,  k) 

~  Qi,b(i,k)  Qi,k  ^  Oq. 

Consider  an  example,  where  the  control  is  over  either 
power,  bandwidth,  or  time.  Let  the  rates  be  propor¬ 
tional  to  the  allocations  Bj  f,,  with  constants  of  propor¬ 
tionality  cl  J,.  The  rate  is  qh,  =  k-  There  are  the 

resource  constraints  ^  for  each  j,  k,  and  the 

mean  throughput  constraints  >  A“,  all  k. 

Any  solution  ql  satisfies  (A2.4). 

III.  Liapunov  Function  Perturbations  and 
Proof 

The  perturbations.  We  now  define  the  Liapunov 
function  perturbation  SV (n) .  This  will  be  a  sum  of 
terms,  one  corresponding  to  each  input  process  one  to 
each  output  process  of  each  queue.  The  motivation  for 
the  structure  of  the  perturbations  should  be  apparent 
from  the  way  that  they  are  used  in  the  proof.  Addi¬ 
tional  background,  applications  and  motivation  are  in 
[7],  [9].  Recall  that  k{i)  =  arrival  node  for  data  from 
source  i.  The  “arrival”  perturbations  are,  for  k  =  k{i) 
and  some  integer  to  >  0, 

n+m—  1 

^  [a,.fe(0  -  A^^]  , 

l—n 

(3.1) 

The  value  of  to  will  be  chosen  in  the  proof.  Set 

SV^l^in)  =0ifk^  fc(*). 

Recall  that  the  vector-valued  channel  state  j  denotes 
the  canonical  state  of  the  set  of  channels  on  the  forward 
links  from  the  node  in  question.  In  (3.2),  we  define  two 


sets  of  perturbations.  The  top  one  is  concerned  with  the 
effects  of  the  departure  of  packets  from  queue  (i,  k)  on 
the  value  of  n) ,  under  channel  state 

j,  and  the  “reference”  rates  qf of  (A2.4).  The  bottom 
one  is  concerned  with  the  effects  on  this  value  of  the 
inputs  to  (i,  k)  from  queue  (i,  b{i,  k)),  when  the  channel 
state  at  node  b{i,  k)  is  j,  and  under  the  “reference”  rates 

<K3^n)  = 

n+m—  1 

~Wi^kXfi^  {ri)qif.  En  [l{L^{i)=j}  —  Bkj]  , 

l—n 

n+m—  1 

'^i,kXff.  (R)9{f,(i,fc)  ^{Lb{i,k){l)=3}  ~^b(i,k),j  ■ 

l—n 

(3.2) 

The  full  Liapunov  function  perturbation  5V (n)  and  the 
time-dependent  Liapunov  function  V  (n)  are 

5F(n)  =  ^<5F+(n)+  ^ 

i,k  (3.3) 

V{n)  =  V{X{n))  +  6V{n). 

Theorem  3.1.  Under  (A2.1)-(A2.4)  the  system  is  sta¬ 
ble. 

Proof.  V (n)  is  the  (time- varying)  Liapunov  function 
that  is  to  be  used.  We  need  to  show  that  there  is  c  >  0 
such  that  EnV{n-\-l)  —  V{n)  <  —  c  when  \X{n)\  is  large 
enough,  and  then  that  this  inequality  together  with  the 
bounds  on  SV{n)  imply  (2.1). 

As  usual  in  stability  proofs,  one  must  evaluate 

E„V{n-\-  1)  -V{n)  =  '^wt^kEn  A+(n-|-  1)  -  A:+(n) 

i,k 

+  ^  +  1)  — 

i,k 

+  E  En[svf^,%+l)-6Vf^,^^{n)  . 

This  will  be  done  component  by  component,  and  then 
the  results  added.  This  will  have  the  effect  of  either 
“averaging”  undesirable  terms  or  else  suitably  domi¬ 
nating  them.  This  is  the  key  to  the  effectiveness  of  the 
method.  A  first  order  Taylor  expansion  yields 

E  ^rkEn  [++(n  +  1)  -  A+(n)]  =  0(++"(n)) 

i,k 

^  ^  (^)  “1“  (^)]  ■ 
i,k 

(3.4) 

Now  consider  the  “arrival”  perturbation  component 
(3.1).  If  fc  =  k(i),  then 

A+F+(n+l)-<IF+(n)  = 

~'^i,kX^j,  (n)  \EnO,i,k{n)  —  Aj  ;.] 


where 


plus  error  terms  bounded  by 


e“fcW  =  Wi^kXl^'^in)  \Ena,^k(n  +  m)  -  A“fc| 

+mWi^kO{Xf^‘^{n))  (3.5) 

^  w^,kXf^^{n)p{m)  +  mO{XP~‘^{n)). 

Thus,  summing  over  i,k, 

Y,En[SV^k{n+l)-SV::k{n)] 

i^k  / Q 

i,k 

plus  error  terms  bounded  by 

J2^i^>^^lk\ri)p{m)  +  mJ20{Xl';:^{n)).  (3.7) 

2,fc  i,k 

By  adding  (3.4)  and  (3.6),  the  Wi^kXf^^ {n)Enai^k{n) 
terms  are  replaced  by  the  mean  value  term 
Wi  ,kX!-k^  (n)A“^  term  and  an  “error”  term.  The  er¬ 
ror  term  will  be  dominated  by  the  main  terms  of  order 
p  —  1  for  large  values  of  the  queue  state  and  appropriate 
values  of  m.  Such  replacements  are  the  motivation  for 
the  form  of  the  perturbation  (3.1). 

Now  deal  with  the  top  term  in  (3.2).  This  will  help  to 
“average”  the  di^k{n)  term  in  (3.4).  By  the  definitions. 


En 

n+m 

—w, 

bkEnXfj,^{n+  l)ql^ 

fe  E 

En+l 

[l{Lk{l)=j} 

/— n+l 

n+m— 

1 

-l-w, 

E 

l—n 

E'n  \_^{Lk(l)  = 

=1}  ~  Hfcj] 

This  expression  can  be  written  as 


-n 


k,j\ 


(3.8) 


[hLk{n)=j}  -  Hfej 


n+m 

—  Wi^kEnXfj,  {n+l)qij,  En+l  [l{Lk{l)=j}  ~^k,j] 

/— n+1 

n+m—  1 

+w^,kXfj:^{n)qlf^  En  [l{L^ii)=j}  -  Hfej]  . 

/— n+1 

(3.9) 

Writing  V^y(n  -f  1)  =  Xl^\n)  +  [^^^(n  -f  1)  - 
xf-k^  (n)]  and  expanding  the  bracketed  term  yields 


(3.10) 


En[SV:^Cj(^+l)-dV:^i^^{n)\ 

=  '^i,kXfj,  i'njqjj^  [kLk{ri)=j}  ~  life  j] 
plus  error  terms  that  are  bounded  by 
^i,kdikXlk\n)  \E^I{LUn+m)=j}  “  ^k,j\+m  (u)) . 

Analogously,  one  can  show  that 


E„ 


SVf^Qin  +  l)-SVf^Q{n) 


kLb(i,k){n)=j}  ^b{i,k),j 


(3.11) 


'^i,kQi^b(i,k)Xpk  M  \EnI{Li,(i^k)in+m)=j}  ^b(i,k),j 

rP- 
'^i,k 


+mO{XPf{n)). 


Adding  all  terms  in  (3.4),  (3.6),  (3.10),  (3.11),  plus 
the  error  terms,  and  cancelling  where  possible  yields 


EnV{n+l)  -V{n)  =  ^  Wi^feA^?^  ^(n)A“fe 


i,k 

i,fe(n)  +  Wj,feV?^.^(n)di_b(i_fe)(n) 
i,k 

+  ^1]  '^i,kXfk  {'E)qi^k  \.^{Lk(n)=j}  ~  Ilfcj] 

i,k,j 

i,k,j 

+  error  terms  bounded  by  (2  -|-  2Kq)x{3.7). 

(3.12) 

Separate  out  the  terms  in  the  middle  three  lines  of 

(3.12)  that  do  not  involve  the  life  j  variables,  getting 


^{Lb(i,k)in)=j}  n&(i,fc)j 


-  X!  '^hkXlk^in)  [di,fc(n)  -  di^b{i,k){n)] 

k 

[^ikhLkin)=j} 

i,k  j 

—  y]  '^i,kX^^.  (n)  y]  [9j,h(i,fe)-I{it(i,).)(n)=i} 

i,k  j 

The  last  expression  can  be  written  as 

^  ^  ^i.kX^  j^  {ti)  \di^kikl)  dj fe^  (?T-)] 


i.k 


-k^w*,fcA^y^(n) 

i^k 


~Lk{n)  _  ~Lb{i,k)(^n)^  (3.13) 

^i,k  ^i,b{i,k) 


Suppose,  for  the  moment,  that  all  Xi^k{n)  >  Kq. 
Then,  by  (A2.4)  there  is  a  control  {■Ui_fe(-)}  such  that 
under  channel  state  j  the  output  from  queue  {i,  k) 
will  be  ql,,  =  gi^k{j,Xi^k{n),upk{3,X{n))).  Since  the 
di^k(n)  are  chosen  by  one  of  the  maximization  rules 
(2.3),  (2.4),  or  (2.5),  and  the  qlf.  outputs  defined  in 
(A2.4)  are  not  necessarily  maximizers,  the  expression 

(3.13)  is  non-positive.  Using  this  fact  in  (3.12)  together 
with  the  definition  of  qi^k  in  (A2.4)  yields  the  upper 
bound  to  (3.12): 

'y  ^i,kX^  i,  (n)  [Aj  fe  —  di^k  +  di,b(i,k)\ 

i,k 

+  {2  +  2Kq)  Z«,..,xrp(nMn,)  +  ,„Zo(xlpM). 

i,k  i,k 

(3.14) 

By  (2.6),  A“fe  -  -k  i  k)  <  — cq.  Select  the  integer 
m  so  that  (2 -|- 2Aro)p(m)  <  co/2.  Thus,  since  m  is  now 


fixed, 

E^V{n+l)-V{n) 

<  -[co/2]  +  0{\X{n)r^). 

i,k 

(3.15) 

We  also  have 

\5V{n)\  =  0{\X{n)r^)  (3.16) 

and,  by  (3.15), 

EnV (n  +  1)  —  1^ (n)  ^  — oo,  unif.  in  n  as  X{n)  oo. 

(3.17) 

By  (3.17),  there  are  ci  >  0  and  qo  >  0,  such  that,  for 

\X{n)\  >  qo, 

EnV{n+l) -V{n)  < -Cl.  (3.18) 

Given  small  (5  >  0,  (3.16)  implies  that  for  qo  sufficiently 
large, 

\V{X{n))  -  V{n)\  <  (5(1  +  V{X{n)).  (3.19) 

Let  (To  be  a  stopping  time  for  which  |X(cto)|  =  C2  >  qo, 
and  define  ai  =  min{n  >  cto  :  |-^(n.)|  <  (jo}-  Then,  by 
(3.18),  we  have 

EaoV{ai)  -V{ao)  <  -ciE,jg[ai  -cto].  (3.20) 

Using  (3.20)  and  the  bound  (3.19)  on  V{n)  —  V{X{n)) 
to  bound  U((Ti)  —  V{X{ai)),  i  =  0, 1,  yields 

-6E,,„[1  +  U(X((Ti))]  +  E,,,ViX{ai))  <  E„,V{ai) 

<  —ciEa-gicfi  —  CTo)  +  [(5  +  U(X((Jo))(l  +  (5)] 

or 

E,  (ai-ao)  <  ^  +  ^)  + 

which  implies  that  the  definition  of  stability  (2.1)  holds 
since  U(X(cri))  <  sup|,,|<,j,  U(a:). 

There  is  no  space  to  complete  the  details  when  some 
components  of  X{n)  are  less  than  Ko-  The  required 
adjustments  are  minor  and  it  can  be  shown,  similarly 
to  what  was  done  in  [8] ,  that  the  contributions  of  those 
terms  are  bounded,  so  that  the  contributions  for  large 
Xi.kin)  dominate.  The  details  are  omitted.  ■ 

Comments.  The  rule  (2.3)  requires  that  each  node  k 
know  the  value  of  the  Xi^k{n)  and  Xi  f(^i  k){n)  for  all  i 
that  use  node  k.  It  is  easily  seen  from  the  proof  that 
the  value  of  Xj  j(j  j,)(n)  need  only  be  known  approxi¬ 
mately  at  node  k.  The  quantities  qo,  S,  ci  all  depend  on 
the  values  of  the  p(-)  in  (A2.2)  and  (A2.3)  and  on  the 
excess  capacity  of  the  system  as  quantified  by  cq.  If  the 
rate  of  mixing  of  the  channel  process  is  very  slow,  then 
the  queues  will  often  have  very  large  excursions,  de¬ 
spite  the  fact  of  stability.  Consider  the  possibility  that 


some  links  are  preempted  by  priority  users  from  time  to 
time,  where  the  intervals  of  availability  are  defined  by 
a  renewal  process  that  is  independent  of  the  arrival  and 
channel  rate  processes.  Then  it  can  be  shown  that  the 
results  continue  to  hold,  but  with  the  qi^k  multiplied  by 
the  fraction  of  time  that  the  channel  is  available,  so  the 
capacity  must  be  sufficient  to  handle  the  down  times. 
Under  the  other  assumptions,  (A2.4)  is  sufficient  but 
not  necessary  for  stability,  but  it  is  “nearly”  necessary 
in  the  following  sense.  Suppose  that  for  each  allowable 
choice  of  the  {9/^,},  there  is  some  {io,ko)  such  that 
qio,b{io,ko)  -  Qio.ko  >  0-  Then  the  system  is  not  stable. 

IV.  Heavy  Tailed  Input  Distributions 

Poisson  arrival  processes.  Suppose  that  source  i 
generates  input  files  whose  lengths  have  heavy  tailed 
distributions  in  that  their  second  moment  is  finite  but 
for  some  0  <  (5  <  1  the  (1  -I-  (5)th  moment  is  finite.  Let 
the  file  creation  process  be  Poisson  with  rate  Ci/slot 
with  mean  file  size  fhi.  Thus  the  mean  input  rate/slot 
is  di  =  Cifhi.  Suppose  that  successive  files  are  mutually 
independent,  and  that  the  file  size  is  known  when  cre¬ 
ated®  and  that  the  file  is  sent  to  the  system  at  a  rate 
A“  >  di  packets/slot.  [Note  the  new  definition  of  A“.] 
Use  the  following  slightly  revised  model  of  the  system. 
Introduce  a  fictitious  node,  called  p{i).  Suppose  that 
the  entire  contents  of  any  new  file  from  source  i  is  sent 
all  at  once  to  this  node,  and  then  sent  from  this  node 
to  the  true  entry  node  k{i)  of  the  network  at  a  rate  A“, 
if  queue  p{i)  is  not  empty.  This  is  the  source  i  process 
that  would  have  gone  to  node  k{i)  directly  if  node  p{i) 
were  not  introduced,  as  in  the  previous  sections. 

Augment  X{n)  by  the  new  state  Xpi^i){n)  and  add 
the  term  [1  -|-  Ap(q]^+®/(l  -|-  6)  to  V{X).  This  term 
will  not  affect  the  decision  rule  since  the  output  from 
node  p{i)  is  not  controllable.  From  the  point  of  view 
of  the  proof,  the  input  rate  to  node  k{i)  from  p{i)  is 
constant  at  A“  if  Ap(q(n)  >  0.  Suppose,  w.l.o.g.,  that 
the  input  file  size  is  always  greater  than  A“  and  is  an 
integral  multiple  of  A“. 

The  only  new  contribution  to  the  proof  is  the  behav¬ 
ior  of  (divided  by  1  -I-  5) 

A„[l  +  Ap(q(n  +  1)]^+^  -  A„[l  +  Xp(i)(n)]®+®.  (4.1) 
This  can  be  written  as 

[1  T  T  ^p(i)(^))] 

X  {ap(z){n)  -  dpE){n))  , 

(4.2) 

where  0i{n)  is  a  random  variable  with  values  in  [0, 1], 
and  dp(^i'f(ji)  =  A“  if  the  queue  is  not  empty.  In  the 
event  that  there  is  no  arrival  at  n,  by  the  monotonicity 

®These  file  sizes  will  not  be  used  by  the  decision  rule.  It  is  only 
required  that  they  be  included  in  En, 


of  (1  +  we  can  bound  (4.2)  from  above  by 

-dp(i)  (n)  [1  +  (n)]  .  (4.3) 

In  the  event  that  there  is  an  arrival  at  n,  treating  the 
two  components  due  to  the  arrivals  and  departures  sep¬ 
arately  and  using  (a  +  b)^  <0*^-1-  ,  (4.2)  can  be 

bounded  above  by 

~dp(i)(n)  [l -|- (n)] 

s  a  (^■^) 

-h  [l -f  Xpp)(n)]  Th^  +  E\ap(^){n)\^^  . 

To  get  the  desired  bound,  weigh  (4.3)  by  (1  —  q)  and 
(4.4)  by  Ci-  Repeat  the  above  procedure  for  each  heavy 
tailed  input.  The  rest  of  the  proof  is  as  for  Theorem  3.1. 
One  only  needs  to  keep  track  of  the  different  powers  of 
the  Xi^kin).  The  condition  (A2.2)  need  hold  only  for 
the  non-heavy=tailed  inputs.  No  additional  perturba¬ 
tions  are  needed,  due  to  the  Poisson  and  i.i.d  property 
of  the  input  sequence  ap(i)(n). 

Renewal  arrival  processes.  Continue  to  suppose 
that  the  heavy  tailed  arrival  file  sizes  from  source  i  are 
i.i.d.,  with  mean  fhi.  But  now,  let  the  arrival  times 
constitute  a  random  point  process.  Let  If{l)  denote 
the  indicator  function  of  an  arrival  at  time  I,  and  sup¬ 
pose  that,  for  some  q  >  0,  |i?„/f(l)  —  Ci\  <  p{l  —  n) 
for  all  n,l  >  n,u),  where  p{n)  ^  0  as  n  ^  oo.  Then 
the  proof,  modified  as  above,  goes  through,  with  the 
additional  perturbation 
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n-\-m—  1 

l—n 

(4.5) 

where  the  integer  m  is  chosen  as  in  the  proof.  Next, 
suppose  that  the  file  sizes  are  correlated  and  that 
|if„/“(/)ap(j)(l)  —  di\  <  p{l  —  n)  for  all  n,  I  >  n,uj.  then 
the  modified  proof  goes  through,  with  use  of  the  per¬ 
turbation 

dVp^i)in) 

n-\-m—  1 

=  [1  +  Xp(^,){n)Y  [A„/“(0ap(q(0  -  d*]. 

l—n 

(4.6) 
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